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ABSTRACT 

The inappropriateness of using standardized 
achievement tests to measure the cognitive skills of Navajo children 
was demonstrated in a 1975-76 case study of 17 Navajo preschool and 
daycare children in Los Angeles. The Circus Receptive Vocabulary Test 
and the Circus Quantitative Concept Test were given individually to 
each .child. Scores were compared with a sample of 66 randomly drawn 
children in the Tribal American preschool and daycare programs, and 
with national results. On both tests, the Navajo children in the 
study scored the lowest, with results below the national average. 
Children who had most difficulty with the tests were the youngest and 
lived in the least acciilturated Navajo households, suggesting that 
the acculturat ive status of the child's household and the age of the 
child influenced the child's response to the testing situation. The 
study concluded that because complex behavioral interactions involved 
in testing situations are almost never reported, "objective" tests 
results take on a reality that does not exist. Testing all 
individuals with the same instrument regardless of 
individual/cultural backgrounds is a misleading and culture-bound 
method of assessing and understanding the abilities of any given 
individual. (NEC) 
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ABSTRACT 

The inappropriateness of using standardized achievement tests to 
measure the cognitive skills of Navajb. children is described, A case 
study is discussed in which two types of achievement tests were used to 
assess the vocabulary and quantitative skills of seventeen Navajo pre- 
school and daycare children in Los Angeles, The test instruments and 
testing procedures are described. The f, objective n results are presented. 
And, the unusual behavioral reactions of four Navajo children to the 
testing situation are described and analyzed. Finally, the test results 
are critically examined leading to a discussion of the usefulness and 
accuracy of the test results. 
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The purpose of this paper is to describe the inappropriateness of 
using standardized achievement tests to measure the cognitive skills of 
Navajo children. Tbfis will be accomplished by describing the problems 
encountered in administering the Circus Receptive Vocabulary Test and 
the Circus Quantitative Concepts Test to seventeen Navajo and seven Cau- 
casian preschool and daycare children in Los Angeles. After the test 
instruments and testing procedures have been described, the "objective" 
results will be presented in the typical statistical manner. Then, the 
unusual behavioral reactions of four Navajo children to the testing situa- 
tion will be described leading to a critical discussion of the usefulness 
and accuracy of the test results . 

RESEARCH CONTEXT 

This study was conducted during the 1975-1976 school year in the pre- 
school and daycare classrooms and playgrounds of the Tribal American Con- 
sulting Corporation (TACC), an Indian-operated, community-action organiza- 
tion located in the Los Angeles area. TACC provided a number of services 
to 210 Indian children and their families. A smaller number of Caucasian 
and Mexican-American children also received services from this corporation. 

The children in Tribal American were distributed in ethnically mixed 
classrooms and playgrounds. The sex and ethnicity of the teachers and 
teachers' aides varied from classroom to classroom and from playground 
to playground, but there was at least one Native American instructor in 
each setting. During the research, the sex and ethnicity of the instruc- 
tors were: one Caucasian male, five Caucasian females, one Native Ameri- 
can male and seven Native American females. Three of the Native American 
females we're Navajo. 



SAMPLE SELECTION 

Sixteen children were enrolled in the 1975-1976 school year whose 
father and mother reported themselves ac being "full blood n or "4/4" 
Navajo. One more child was present whose mother was "4/4" Navajo and 
whose father was both unlisted and absent from the home. The average 
age of the seventeen Navajo children in the sample was four-years three- 
months. Eight were males and nine were females. 

Parents of thirteen of the seventeen Navajo children were asked to 
assess by questionnaire the speech patterns in their homes. These parents 
reported that they frequently spoke Navajo in the home. They also reported 
that they spoke Navajo in the home to other adults much more frequently 

than they spoke Navajo to their children. Finally, all but one question- 

7" 

naire reported that the children of interest in the household spoke Nava- 
jo in the home "not very often" or "never." Several parents described by 
personal interview their reluctance to speak Navajo to their children. 
They believed that speaking Navajo to their children interfered with the 
process through which their children learned English. Unfortunately, 
this questionnaire data may not represent accurately the sociol inguistic 
situation in the homes. Indeed, Bernard and Killworth (1977) have shown 
that in communications research informant cognition of interaction is not 
an accurate indicator of actual interactive behavior. 

A list of all the Caucasian children in the preschool and daycare 
programs was also compiled, Only 7 children were reported a* being com- 
pletely Caucasian. Two of these children had no father presently living at 
home. Four were females and three were males. Their average age was four- 
years one-month. 
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This research was intended to be a detailed study of a specific 
school situation. Knowing that the sample sizes are rather small, parti- 
cularly for the Caucasian children, we should be cautious when extending 
conclusions based on this research to other Navajo and Caucasian popula- 
tions. Since the methodology is replicable, direct comparisons between 
these research populations and others are possible. 

RESEARCH METHODS 

The American Indian Education and Early Childhood Education funding 
grants which supported Tribal American during this period stipulated that 
the children should be given achievement tests assessing their vocabulary 
and numerical skills. Upon the request of TACC personnel, this researcher 
volunteered to administer the Circus Receptive Vocabulary Test and the 
Circus Quantitative Concepts Test, created by the Educational Testing 
Service, to the seventeen Navajo and seven Caucasian children. Each 
child was tested individually. Only one test was given each day and no 
time limit was set. 

The children were familiar with me by the time I administered the 
test for I had been informally observing and participating in classroom 
and playground activities for at least a month prior to testing (Guilmet, 
1981, 1979a, 1979b, 1978). I would walk into the classroom, call a child's 
name, and take him/her to a separate room where the test was given, free 
from the noise of the classrooms. Sope of the Navajo children seemed glad 
to be tested, but most showed little emotion other than complete silence. 
I tried to talk as softly as possible and to reassure each child. 

These Circus tests are in no way culture-fair nor do they pretend to 
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be. They say nothing about intell igence- or any related construct. How- 
ever, they are considered by many educators to be s a measure of the voca-. 
bulary and numerical performances of a child in a testing situation when 
English is the language used for communication. These tests are used by 
educators in determining the extent to which a child is prepared to suc- 
ceed according to Western standards in public schools. 

In the vocabulary test the child is presented with a piece of paper 
with three pictures on it; for example, a picture of'an ear, a person's _ 
heel and a person's toes. The child is asked to mark with a crayon the 
picture of the heel. Similarly^ a picture of a girl petting a dog, hold- 
ing a dog and throwing a ball to a dog is presented to the child. The 
child is asked in this case to mark the picture of the girl holding the 
dog . 

The quantitative test also presents the child with a piece of paper 
with three pictures on it. The child is likewise aiked to mark one of 
the choices. For example, the child is presented with a picture of one 
lion, a picture of three lions, and a picture of five lions and he/she 
is asked to mark the picture showing three lions. Or, two groups of balls 
are shown to the child and rfe/she is asked to mark the group that has more 
bal Is. 

RESULTS 

The mean score for the seventeen Navajo children on the Circus Recep- 
tive Vocabulary Test was 14.59. The mean score for the seven Caucasian 
children on the same test was 20.57. A sample of 65 randomly drawn chil- 
dren in the Tribal American preschool and daycare programs, tested in the 
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same time period, scored 17.21 on this vocabulary test. Nationwide, 
children averaging 4 years and 9 months scored a mean of 27.8. The 
maximum possible score on the test is 40.0 

The mean score for the seventeen Navajo children on the Circus Quan- 
titative Concepts Test was 15.41. The mean score for the seven Cauca- 
sian children on this quantitative test was 22.71. A sample of 65 ran- 
domly drawn children in the Tribal American preschool and daycare programs, 
tested in the same time period, scored 19.18 on this test. Nationwide, 
children averaging 4 years and 9 months scored a mean of 28.1. The maxi- 
mum possible sco^.e 1 is 40.0. 

The patterns on the two tests were the same. The Navajo children 
scored the lowest.' The random population of Tribal American children 
scored the next highest. The Caucasian children in Tribal American scored 
higher than the first two groups but lower than the average for the nation. 
The mean ages of the Navajo and Caucasian samples are respectively six 
and eight months younger than the mean age of the nationwide sample. 
However, the random population of Tribal American children tested in Octo- 
ber showed no significant change in performance on either test when re- 
tested in January. Thus, the patterns on the two tests mentioned above 
would probably be maintained if the mean ages of the test groups were the 
same . 

The seven Caucasian children are from low-income households. To be 
eligible for state and federally funded daycare and preschool programs the 
income of the child's parents must be comparatively small. These test re- 
sults are consistent with the general finding that children from low 
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socioeconomic-status homes score lower on standardized tests than chil- 
dren of the same age from middle- and high-inccme families. 

DISCUSSION 

The best way to give context and meaning to the low scores of the 
Navajo children is to describe the kinds of experiences I encountered 
while admistering these tests. Consider the interesting behavioral re- 
actions of the following four Navajo children to v the testing situation. 

Caryn (3 years and 11 months) was absolutely quiet pr,ior to testing. 
She watched me closely, did not appear afraid or happy but mildly curi- 
ous. This was her second year in Tribal American, so she had been tested 
at least three times last year. Caryn was unique in that each time I 
would ask her to do something, she would repeat my statement. For example 
when I said "Mark the picture of the biggest monkey" she said "maak da 
beegest munkey." Her repetition was rarely complete, nor was it phoneti- 
cally the same as mine. After repeating my statement she would randomly, 
or almost randomly, mark one of the three pictures* in front of her. Over 
and over through two tests and eighty questions she would repeat my state- 
ment and randomly mark a picture. Caryn also had a hard time staying 
on task; she would look around the room, then at me, then at the pencil, 
etc. She never initiated conversation, only repeated. 

Sherry (2 years and 10 months) is Caryn's younger sister. She was 
neither afraid nor happy, but . probably more hesitant to be tested than 
her sister. This was her first year at Tribal American and her first ex- 
posure to a testing situation. Sherry's response was absolute silence 
from the time of 1 saving the class until she returned. I could not get 
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her to understand the idea of testing. She repeatedly took the crayon 
in hand and drew a line through all three choices. No matter how I 
tried to explain to her the task, she simply drew her line. Near the end 
pf each test, I marked the picture where she started her line. This is 
the only reason she registered any score at all, 

9 

Pat (2 years and 10 months) did not utter a word during the testing. 
Rather, he sat in his chair and watched me. He did not respond in any 
way to anything I did, I could not get him to pick up the crayon or 
mark one, picture in either testing session. He did not appear to be too 
afraid or mad, but he did appear to be unsure of the situation. This was 
his first year at the school and probably his first experience with a 
testing situation. 

Tony (4 years and 2 months) is Pat's olde^ brother. During the test- 
ing sessions I asked him some questions but he did not respond, Tony 
did not talk at any time during the tests even though this was his second 
year in the preschool. In contrast to his younger brother, Tony had prior 
exposure to testing and knew that he should make a choice between the pic- 
tures given him. He was so eager to get finished with the tests that he 
never waited for the instructions to be given, I would only get as far 
as M mark the , , ." and he would mark it. Near the end of each test I 
held his crayon until the instructions were given, I never knew whether 
Tony understood what was expected or not. 

The above children were among the quietest in individualized work 
periods in classroom situations (Guilmet, 1978) and on the playground 
(Guilmet, 1981), These children spent a great deal of time involved in 
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object play in the classroom. They attended visually to objects, in- 
structors and other students much more often than the other Navajo chil- 
dren. These children rarely spoke even if spoken to by an instructor 
or another child. If they spoke th^y did so in English. Their physi- 
cal motions were deliberately slow and reserved. When watching student- 
student, student-instructor, and instructor-instructor interactions, 
they appeared to be studying them to learn how to participate in social 
( activities. 

A Navajo college student at the University of California at Los . 
Angeles, a fluent Navajo speaker, volunteered to help administer these 
tests to some of. the other children in Tribal American. I asked her to 
talk with Sherry, Pat and Tony to see if she could get them to speak up. 
She tried speaking to them in both Navajo and English, but she had no 
more luck in getting them to communicate than I did. They simply would 
not speak. I asked her if she would retest the Navajo children, giving 
the instructions in Navajo. She refused, stating that there was no i.eed 
for such a study. She said that these children needed enhanced training 
in English in order to be able to do well in school, not more studies 
showing that they could perform tests in Navajo. I would have liked to 
have designed a test which measured their vocabulary skills in Navajo. 
However, her point was well-taken, and I did not push the subject. Neither 
I nor the teachers had ever heard, nor sver would hear in the entire year, 
a child spontaneously speak Navajo either in class or on the playground. 

Not all the Navajo children had as many problems with the testing 
situation as the above students. Several Navajo children were quite tal ka- 
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tive in English, appeared to understand the testing procedures, and 
achieved higher scores by responding "appropriately" and quite accurately 
to each task presented. These children talked much more frequently to 
instructors and students during individualized work periods in classroom 
situations (Guilmet, 1978) and on the playground (Guilmet, 1981). These 
v children much more frequently initiated conversations to instructors and 
students and changed tasks frequently, leaving work unfinished. Indeed, 
some of these Navajo students assumed an assertive leadership role in or- 
ganizing student activities. These children's physical motions were rapid 
and they often verbally demanded instructor attention when they desired it. 

It appears that two factors influence the child's response to the 
testing situation: the accul turative status of the child's household and 
the age of the child. Let us consider each of these variables in turn. 

Based on several indices proven useful for assessing the accul tura- 
tive status of American Indians in Los Angeles (Price, 1968), the four 
children who had difficulty with the tests lived in two of the least 
acculturated Navajo households of the children in Tribal American. Caryn's 
and Sherry's mother was much more reserved in the presence of strangers, 
particularly Caucasians, than most of the Navajo parents I met. She was 
deeply involved in making and selling traditional crafts, and associated 
primarily with other Indians, especially other Nava jos . During their six 
years in Los Angeles Caryn's and Sherry's parents visited their families 
on isolated parts of the reservation almost every vacation. They occasion- 
ally attended the American Indian Peyote Church during these visits and 
both speak Navajo fluently. 
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Pat and Tony's family frequently visited the reservation during the 
beginning of the school year. Having recently moved to Los Angeles from 
the reservation, the parents were experiencing difficulty in adjusting to 
the urban environment. Indeed, Pat and Tony "ere eventually removed from 
Tribal American when their family returned to the reservation to live. 
Both parents were fluent speakers of Navajo and associated predominantly 
with other Navajos. 

In contrast, the Navajo children who had little difficulty with the 
testing situation lived in the most acculturated Navajo households of the 
children in Tribal American. The parents of these children had lived 
in Los Angeles longer, were less involved in reservation life, formed 
wider peer groups including non-Navajos and non-Indians, and were less 
involved in the traditional Navajo religion. 

The second factor influencing test performance was the age of the 
child. The older the Navajo child the more "appropriate" his/her response 
and the higher his/her score. The Navajo- children 's ages are positively 
correlated with their numerical scores on the Receptive Vocabulary Test 
(n = 17, r = .878, p = .0001) and the Quantitative Concepts Test (n = 17, 
r = .843, p = .0001). 1 It appears that the older the Navajo child, the 
less his/her fear of the testing situation, the greater his/her under- 
standing of testing procedures and the greater his/her vocabulary and 
numerical skills in English. 

CONCLUSIONS 

The major lesson to be learned from this experience is clear. The 
"objective" test results take on a reality that does not exist because the 
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complex behavioral interactions involved in the testing situation are 
almost never reported. Thus, the meaning or lack of meaning of the 
test results are obscured by a simplistic data reporting procedure. 

The experience illustrates a key methodological problem in cross- 
cultural research: control of data versus understanding of the meaning 
of observed behavior (Edgerton and Langness, 1974). By testing all indi 
viduals with the same instrument no matter what their individual or cul- 
tural backgrounds, the tester's task is made more simple. He/she need 
only scale each individual according to his/her "objective" score and 
treat each individual according to a prescribed stereotypical pattern 
which corresponds to the subject's "abilities." While this may be con- 
venient for the tester and the institution he/she represents, it is a 
misleading and culture-bound method of assessing and understanding the 
abilities of any given individual. For example, see this author's re- 
view of the problems of utilizing cognitive tests among the Eskimo (Guil 
met, 1975). 

The Bureau of Indian Affairs Task Force on Testing (Blanchard, 1972 
134) recommended the following two policy guidelines: 

1, That standardized tests developed on population norms, 
having as their primary purpose the ranking of students 
oji inferential scales so they may be compared with one 

^another, should be phased out in an orderly but firm manner, 

2, That criterion-reference tests tied to curriculum con- 
tent and integral with educational and behavioral objec- 
tives become the tests of choice. 

In addition, the Task Force noted that standardized tests are es- 
pecially harmful to Native Americans because norm-reference tests un- 
fairly discriminate against minority groups. They lead to harmful and 
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inappropriate stereotyping, and are psychologically harmful as traumatic 
experiences to the student. Standardized tests also create artificial 
and unnecessary barriers among students by creating a sense of competi- 
tion through their ranking and comparing procedures. They become instru- 
ments of forced acculturation by their imposition of test-culture values. 

This author 1 s experience*supports the findings of the Task Force. 
Standardized tests ignore many cultural factors which cause Navajo chil- 
dren from traditional households to behave "inappropriately" in testing 
situations. Mowrer has reported that Navajos typically respond with si- 
lence in unpredictable social situations (Basso, 1970). Surely testing 
is unknown and unpredictable to the traditional Navajo child and accounts 
for the silence I observed. Hall (1959) reported that just plain sitting 
is doing something in Navajo culture. One of the Navajo children in 
my sample (Pat) sat throughout the testing situation. Worth and Adair 
(1973) noted that Navajos avoid face to face eye contact especially with 
outsiders to Navajo culture. Imagine the feelings of a young Navajo child 
in a small room with a relative stranger staring at his/her face to ex- 
plain the testing procedure. 

Cole and Scribner (1974) support this author's view that it is ex- 
tremely difficult to evaluate the sources of variation when individuals 
from another culture do not respond to standardized tests in the same way 
that middle- or upper-class Americans do. Further, in most institutional 
uses of standardized tests, no attempt is made to even consider the sources 
of such variation. Irvine and Carroll (1980) have convincingly argued 
that it is possible to devise forms of testing which will display the 
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ability to accurately assess cognitive abilities across cultural bound- 
aries. However, this author's experience illustrates the fact that 
standardized tests in current use in existing educational institutions 
produce cross-cultural data which is difficult to interpret and culturally 
biased. 
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END NOTES 

1 All correlation coefficients and probabilities in this mono- 
graph were determined by weighted product-moment correlations using the 
Statistical Analysis System, designed and implemented by A.J. Barr and 
J.H. Goodnight, Department of Statistics, North Carolina State University, 
1972. Computing assistance was obtained from the Health Sciences Com- 
puting Facility, UCLA, supported by NIH Special Research Resources Grant 
RR-3. 
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